Search Results: "Manoj Srivastava"

16 August 2006

Manoj Srivastava: Burninating spam

It is funny that people say that no one has written software that does Beysian filtering in order to reject Spam at the SMTP level, just when I felt like writing up my mimedefang setup. I am of the school of thought that the only acceptable method of dealing with spam is to reject it at the SMTP protocol level. To accomplish this, I run crm114 and spamassassin from my mimedefang-filter, which runs as a milter while my sendmail is processing mail. I also do conditional grey listing, more on that below. The reason for running two different Spam detection mechanisms is that while one or the other is often fooled and mis-categorizes mail, the score or confidence level of such mis-categorization is low, and the mechanism that is not fooled enables the combined system to do the right thing in most cases. I tend to quarantine and keep even the messages I reject for about a month, and do spot checks, so out of roughly a 800 mails a day volume, I get about one or two spam messages a day, and did not ever reject a mail I manually would classify as ham in the 6 months n which I double checked every decision taken by my system, and in the spot checks I continue to perform. While this is not as good a record as the author of crm114 reports, it is good enough for me. The grey listing: since I try to be conservative, and opt to lean towards not discarding any email that is legitimate at the expense of not trapping all spam, and since I do not discard any mail I accept unread based on an automated decision, I was getting more Spam than I cared for still. So, I added the grey listing layer – but only for mail that were not ambiguously spam or viruses (which are rejected), or ham, which are accepted for delivery. Most of the Spam that was slipping through came from the grey area. So, I implemented grey listing only for these mails. Grey listing happens on a triplet of the class C domain of the senders IP, the sender, and recipient. A grey listed email encounters processing problems, and is temporarily rejected with a 503 for 30 minutes, and then retries are allowed through for 4 hours, in which case the triplet is white-listed for 35 days (catches monthly announcement messages). Known Spam resets the white-listing, and retrying past the window does too. Doing it this way, known good correspondents suffer no delay, unambiguous mail from strangers flows right through, and I improve my filters based on the results all the time, so my discriminator improves. My filter is available for examination. See also the older posting, like the time honoured tradition of /., this posting is a duplicate.

9 August 2006

Josselin Mouette: Rediscover python packaging

After seeing several developer comments implying that they won't package python software because this has become too complicated, I was hit by the fact python-support had drifted away from its original goal: make the developer's life easier. The initial idea was to simplify dh_python and make it use something that could automatically handle byte-compilation, without the maintainer even noticing it.

Therefore, with the help of Manoj Srivastava's priceless work to clarify the policy and of Pierre Habouzit's background work on supervising the transition, I have improved dh_pysupport to be able to generate correct dependencies. I have also finally added the last missing piece of functionality: private modules needing a specific python version.

This is a necessary step for Joey Hess' plan to remove dh_python from debhelper. But the more important thing is, for most packages, dh_pysupport can now be a drop-in replacement for the old dh_python. This means that many remaining packages for the transition can be updated with s/dh_python/dh_pysupport/ and a new build-depend.

And, for anyone who's afraid of maintaining a python package because it is too complicated, let me show how to make a simple package using python:

Yes, that's all. You mean, just like with the old dh_python, the one that used to work without having to understand anything? Yes, just like this one. Don't add a XS-Python-Version field, it's not needed. Don't add a XB-Python-Version field, it's useless. Don't add a debian/pycompat file, it's deprecated. You can add a debian/pyversions file if the package has requirements on the python version, but it's not needed for many packages.

Of course, packages with binary extensions should build them for all available python versions, but dh_pysupport handles both cases. Of course, I don't claim it's free from bugs, but bugs can be fixed now there is a good design.

So finally, I can say it again: python packaging is simple. You shouldn't be afraid of it.

2 August 2006

Isaac Clerencia: Debconf6

After a bunch of trains, subways, planes, buses and taxis I’ve finally arrived at Oaxtepec (Mexico) to attend DebConf6, the annual Debian conference. Besides doing some Debian work, having some fun with long-time-no-seen friends and enjoying the swimming pool, there are several interesting talks I’m interested on: Time for lunch here!

28 June 2006

Anthony Towns: On being DPL

It’s almost the end of financial year in Australia, so what better time to take stock? I’m actually a bit surprised at just how much I need to do that – I knew being DPL would involve some new challenges, but it’s barely two months in and I’m already just about in shock. Guess it serves me right for putting “increasing Debian’s tempo” in my platform…
The two other roles I’ve been in that I would have expected to prepare me more have been Debian release manager, and secretary of Linux Australia. They’re both similar in a way – the release manager does a lot of cat herding within Debian, and has to make some calls that will leave people aggrieved; and Linux Australia has some contact with the press, and some reasonably serious projects going on. So a few issues were completely expected: a random initial slashdotting, a press report, an interview or two, some requests for quotes in articles and press releases, or ongoing opposition and arguments from dedicated Debianites who don’t happen to agree with me on some decision. There were other things that were new and interesting though not really surprising. The leader@ mail address turned out to have just as much spam as I expected, though also some unexpected interesting stuff too, including my first ever personally addressed open letter – in this case by Keith Curtis who also wrote the 10,000 bugs away from World Domination article that got slashdotted a while back. A tidbit:
Everyone agrees that Ubuntu couldn’t exist without Debian, but I also believe that Debian is better setup to take Ubuntu where it needs to go. There are hints that the Ubuntu team feels like they brought a pork chop suit to a lions den. Ubuntu’s user base and development team is growing exponentially, but I believe they could get there much faster with more of Debian’s help.
Keith also forwarded an open letter he’d written to Mark Shuttleworth, and had some interesting comments on the whole Java thing. As well, I suddenly started receiving SPI board mail, as Debian’s advisory representative to the SPI board, and started getting a couple of requests for authorising expenditures (Debian UK reimbursing Matthew Garrett for travel costs to the Gnome Advisory Board meeting in 2005) or providing a Debian representative for a meeting or joining the Google Summer of Code. And then there were the cool things I knew absolutely nothing about, like the excellent news Christian Perrier passed on about the launch of Dzongkha Linux, featured in news reports from Bhutan, India, and Australia. Of course, I completely expected that all this would overload me a bit, and I’d end up getting distracted from what I wanted to do, and not being able to keep up with stuff other people wanted me to do. To help with this a little, I delegated Steve McIntyre as, essentially, a co-DPL, and he’s been working on a few things, including the now successful switch of irc.debian.org over to OFTC, and another fun little endeavour that I won’t spoil just yet. Things started getting a bit weird as I was preparing to travel off to debconf in Mexico – with some frustrated comments from damog and Marga and Gunnar, which was odd coming from the conference organisers themselves; and independently the reinflamation of some old tension on the debian-installer team. That got compounded at debconf itself, with the controversy over the inclusion of Sun Java in non-free extending not just to the lists, but an article in LWN (with a followup) as well as numerous other places around the net. That issue came to something of a climax when John Goerzen (with whom I’ve had some entertaining disagreements in the past) questioned whether SPI had been sufficiently involved in the decision (“I am becoming increasingly concerned at the unilateral method in which you and/or the archive maintainers have taken this decision.”), to which I responded fairly curtly to defend the way Debian makes decisions (“If SPI wish to withdraw from their relationship with Debian, then that’s entirely possible to arrange. I don’t think it’s at all proper that you try to obtain veto power of Debian’s activities as conducted by the duly authorised members of that organisation.”). Nothing’s really news there – flamewar on Debian lists between influential project members? Next thing you know the sun will rise in the East every morning, and then where will we be? What’s not so normal is those flamewars getting front page coverage on slashdot, or noted in an article on distrowatch. As far as Debian’s concerned, we had a couple more rounds, both in public and on the developer only -private list, then moved onto actually getting the legal advice John wanted. That’s actually still pending, and the debate has pretty much abated while that’s going on. And meanwhile, Wouter Verhelst, Manoj Srivastava, and others have started a much more productive look at Debian’s relationship with SPI and similar organisations, which has continued on to a draft of a constitutional amendment. But there was more stress and chaos at debconf than just the Java stuff, but what’s really impressive is the way people ended up dealing with it. Take damog for example:
Just as Tore said: This is the best DebConf ever. Isn’t it great? I started really enjoying this DebConf once I stopped worrying about bullshit, once I stopped worrying about senseless things and started to really give a shit about almost anything. Why do I need Gunnar to tell my girlfriend Ana, “you picked up the irresponsible one”? Why do I have to worry about the DPL giving a shit or not on the Latinamerican Debian community, even after being invited to meet the guys or to attend our BOF? […] I enjoy people here, thanks to all the organization cabal, thanks for your effort and your time on this. […] But after leaving all of this behind, I think Ana and me are having a great time sharing time with friends around us (the friends, los cuates, we are here for).
After seeing that post, and figuring out who damog actually was when he gave a lightning talk towards the end of the conference, we had a chance to briefly chat about what was going on – somehow I hadn’t taken in the invitation to attend the BOF, and had been told it was all in Spanish anyway, so hadn’t gone; meanwhile they’ve been thinking about holding a Debian miniconf in Latin America somewhere and wanting to know what sort of support they can get from the rest of the project – particularly moral support rather than financial, at that. Hopefully we’ll see news on that score in the coming months. But really, damog was right in the first place anyway: why care if the DPL supports you or not? Debian’s about everyone doing what they think is good and worthwhile, and that combining to produce something great – it’s not about what some guy who got six votes more than the next guy happens to think is important. But support is important, whether it comes in the form of a DPL saying “good idea!”, or something else. Gunnar’s response to the latter, eg:
Debian is love. After my scream for help a couple of days ago, and after a mountain of hard work, things are just running. No, we are not -by far- free of incidents, and it would be foolish to expect it to be so, but we are working nicely. And by the way, thank you, I have been receiving the largest amount of hugs ever, and believe me, each of them has been important. […] Thank you all, folks. I am in Cristoph Berg’s talk about reworking NM - And this comes very good to wrap up my post. Debian is much, much more than technical work. It is a social club. I love this social club. Just sitting here makes long months of work really worth it. A great hug back to you all!
And in the end we got quite a bit out of debconf6, for example more movement on the forthcoming release, improvements on i18n stuff, a new publicity project, movement on updating python, amongst other things. Who knows if the next debconf will manage to be anywhere near as much of an experience, but at least we’ll find out fairly soon where it’s going to be (though that process isn’t without hiccups of its own). Post debconf, there’s also been a notable influx of trolling going on too; with the most obvious and odious example being the insane, anonymously-posted prayers for a female developer’s death. Fortunately those have mostly been ignored as the mindless spam they are, though one might argue that Marcela Tiznado’s acceptance as a DD on her birthday, and the inclusion of women.debian.org in the official debian.org namespace are a more appropriate response to that sort of harassment than any sort of direct reply anyway. Not all our problems have conveniently been resolved right now, of course, the difficulties related to the installer team I alluded to above are ongoing, and at the moment a really good solution is still eluding us, though development continues anyway. And then, of course, there’s more to come – half the point of posting this is so I can stop worrying about all the stuff that’s been and gone, and get on with interesting things like the next point release, and the various other projects that I’ve let languish for the past few weeks.

11 June 2006

Erich Schubert: SELinux reference policy in Debian unstable

Manoj Srivastava has uploaded reference policy packages to Debian. If you're going to setup a SELinux system with Debian, I'd recommend you to use the new policy packages (which are currently named "selinux-policy-refpolicy-*", but that might be shortened to selinux-policy-* or so sometime, I guess) This is basically the only SELinux policy tree actively maintained. The old policy, packages as selinux-policy-default, is discontinued upstream.

16 May 2006

Ted Walther: Report from Debconf, Day Two

Photography. My roommate Aigars is quite a photographer, he came with his digital SLR and several lenses, an item I've lusted after for some time. Large apertures and manual controls are what every real photographer demands, and digital SLR's deliver. I told him my idea for a small sky recording station so that people could make movies of the paths that the stars take, and how it changes over the year. Ultimately I want a network of these stations, all uploading their pictures so I can examine them for anamolies. This is a hard problem, because stars are so faint. Aigars said to take a picture of the whole sky, a 1000 millimeter lense and a $50 computer scanner with some modifications should be able to do the job.

Me and Andreas Schuldei
Programming. Finally made progress in programming, now that I know where everything is in Oaxtepec. My IRC bot didn't handle things gracefully when the server refused to let it log in, but that is fixed now. Tomorrow I hope to add multi-channel support. Reading through the new IRC RFC, I have the information I need to make a proper configuration file for the software. The design is sketched out and feels right, finally. Weekend vs. Weekday. Oaxtepec is much quieter today. Saturday and Sunday there were far more vendors on the street, and they stayed open much later. Today everything closed down around 6pm when the church bell tolled the summons to mass. I couldn't find the coconut man; maybe he only comes on weekends. Half the stalls at the mercado were empty. During the weekend, the resort was full of vacationers swimming, barbecuing, picnicing, playing soccer, or just sunbathing while the kids ran around. Everyone, young and old, but mostly young adult men and women, were running around in bathing trunks and bikinis. The happy noises were great to wake up to. This morning the sounds were quieter, but still present. The swimming pools here get a lot of use. During the weekend, there was a large tent where some ladies were giving body massages for $10. I didn't see them today. Probably they will be here again next weekend. Food. Tried a dish called "pancita" today. I saw people eating noodles and assumed pancita was a noodle dish, like the Filipino pancit. But no. It is a type of tomato soup filled with chopped up beef rind and chunks of fat. For only P31, I bought a small chicken. It was perfectly marinated and roasted, better than Kentucky Fried Chicken. I paid the money, and the shopkeeper brought out a pair of scissors and cut the chicken into all the appropriate pieces for easy eating, just like they do in Korea (viz the use of scissors) Church. Attended mass tonight, since the church was so close. The church is magnificent. It really is a cathedral inside; to see the ceiling you have to crane your neck. And when you do, you see paintings of angels playing instruments. I went into the nave to pray, and saw a giant depiction of the ark of the covenant. It was very realistic, with the cherubim covering it with their wings. And above it, was a painting of the wine and bread offering, which were offered on the altar daily. The priest was a black man from Illinois; after a bit of attempting to speak to each other in Spanish, we realized we both spoke English, and got on famously from then on. The cathedral opens at 10am, closes for lunch from 2pm-4pm, and then is open until mass is finished. This must be what the church was like in the beginning; a place where believers could drop in any time to be with others of their kind, to meet, eat, drink, discuss, and elevate each other to a higher plane. I am sad that we don't have this in Canada. Debian. There were some Debian talks and presentations, but nothing too interesting. Finally met Mark Shuttleworth, Anthony Towns, Manoj Srivastava, and Roblimo Miller. Clifford Beshers of Linspire almost convinced me to look into the Haskell computer language. I couldn't quite tell what Haskell has that LISP doesn't, apart from forcing strong types on you and being very fast and efficient in benchmarks tests. Marga didn't give me meal tickets before, so Graham printed some up for me today. By pure serendipity, I sat at dinner with the only other Canadian developers at the conference; Simon Law and Eric Dorland from Montreal. The drinking started at 11pm and is still going strong; the atmosphere is very convivial right now, but half of us are still tapping away at our keyboards.

2 January 2006

Manoj Srivastava: In vain, the sage, with retrospective eye –Pope

It has been a tumultuous decade of involvement with Debian for me. I had been on the mailing list since mid 1994, but I was reasonably happy with my SLS system (installed using 40 floppies, including about a dozen for just X11 alone), and while I found Debian intriguing, I was not about to go through the pain of a brand new install until I felt that the new project was viable in the long term :-) I actually jumped ship in the spring of ‘95 and installed 0.93R5. The next step came with Bug 1766, my very first bug report: Bug in script checksecurity in package cron, on 25 Oct 1995. Once in, I rapidly went to the next phase: Here is the sum toto of the NM process I went through: my Hello, World mail. Those were the days :-). There was nothing between my ITP and the upload. My first significant package was kernel-package, since I was always missing something in the series of steps needed to build a kernel, and I started getting into it in the summer of ‘96. This is where the second part of my apprenticeship started – even though I had 3-4 packages in the archive, my kernel images were not yet trusted; so I sent my images to my sponsor (Hi Simon), who then uploaded the images to master. Somewhat later, I also was involved in the early design stages of apt, and the dependency sorting algorithms. While I was fairly silent during the whole DFSG/SC debates (to the extent I was labelled a mindless follower of Bruce –heh), I took an active part in the constitution debates (possibly due to the fallout of the beach story. Anyway, I seemed to want to get us a constitution, after it had seemed stalled for months. It is interesting to note how the technical committee was initially setup, there was a proposal, and then Ian Jackson responded by saying he wanted to appoint the ctte members, since he had been around for a while and was also the DPL. The earliest data I can find is from June of ‘98, as seen in a mail later that year, when the initial list of committee members was created (I also seem to recall I was not on the list initially, but I was added in the early days, before the committee was actually formed). I was interested in the technical policy fairly early, taking a stance that policy was more than just a bunch of ignorable guidelines. Eventually, a [thread that tried to reach a compromise][other] gave us something like the views we (well, I) hold today – including what happens when packages do not follow policy, and about policy editors. And abuse of power. About this time, the policy Czar resigned, burned out, and hounded by accusations of delusion of power. So I first proposed the whole policy editor and consensus approach to letting policy evolve, which eventually led to a formal delegation recently. Brian Basset was our very first secretary. My first involvement with things that would lead on to the secretaries job started with trying to do voting for policy. Then, around 2000, our the secretary and treasurer Darren Benham went MIA, and Raul, as the chair of the technical committee, had to take over and run the DPL election for 2001 – and forgot that DPL elections are supposed to be secret ballots. Ben Collins tapped me for the secretaries position mid-2001, as I recall. It is hard, in a decade or so, to find anything I have not touched – but NM is one such area. Apart from an early pre-current-NM mail, I have not been very involved in NM. Or the Debian installer. Or Debconf. Hmm. I seem to have drifted away from things that Joey Hess is involved in, which is a pity, he is high on the list of people I respect in the project, and this lack of interaction with as time goes on irks me.

Next.

Previous.